Papers with ELMo embeddings
Enhance Robustness of Sequence Labelling with Masked Adversarial Training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Adversarial training (AT) has shown strong regularization effects on deep learning algorithms by introducing small input perturbations to improve model robustness. |
| Approach: | They propose to use adversarial training to improve robustness from contextual information in sequence labelling tasks by masking or replacing some words in the sentence. |
| Outcome: | The proposed method shows significant improvements on accuracy and robustness of sequence labelling on CoNLL 2000 and 2003 benchmarks. |
Gender Bias in Contextualized Word Embeddings (N19-1)
Copied to clipboard
| Challenge: | Existing studies show that training word embeddings in large corpora could lead to encoding societal biases present in these human-produced data. |
| Approach: | They conduct several intrinsic analyses to quantify, analyze and mitigate gender bias exhibited in ELMo’s contextualized word vectors. |
| Outcome: | The proposed method mitigates gender bias on WinoBias probing corpus and demonstrates that it can be implemented in other systems. |
Spot the Odd Man Out: Exploring the Associative Power of Lexical Resources (D18-1)
Copied to clipboard
| Challenge: | Existing word embeddings assign only one vector to each word, resulting in word disambiguation on smaller scales. |
| Approach: | They propose a task which aims to test different properties of word representations. |
| Outcome: | The proposed task is intuitive enough to annotate on a large scale while teasing out properties of popular lexical resources. |
Evaluating Neural Model Robustness for Machine Comprehension (2021.eacl-main)
Copied to clipboard
| Challenge: | evaluating model robustness to adversarial attacks can provide deeper understanding of how deep neural networks work and what kind of linguistic information is actually captured by neural networks. |
| Approach: | They propose a method for strategic sentence-level perturbations to evaluate model robustness to adversarial attacks using character and word perturbations. |
| Outcome: | The proposed model improves model performance during adversarial attacks by using ensembles and predicts errors in adversarials. |
Coloring the Black Box: What Synesthesia Tells Us about Character Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural network models are difficult to understand and are considered "black boxes". |
| Approach: | They use grapheme–color synesthesia to study character embeddings in English . they compare graphemes to phonemes to find the most human-like character embeds . |
| Outcome: | The results show that grapheme-to-phoneme conversion results in the most human-like character embeddings. |
Latent Variable Sentiment Grammar (P19-1)
Copied to clipboard
| Challenge: | Existing neural models do not explicitly model sentiment composition, which requires to encode sentiment class labels. |
| Approach: | They propose a sentiment grammar that captures sentiment subtype expressions by latent variables and Gaussian mixture vectors. |
| Outcome: | The proposed model outperforms vanilla neural encoders on the Stanford Sentiment Treebank benchmark. |
High Quality ELMo Embeddings for Seven Less-Resourced Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent results show that deep neural networks using contextual embeddings outperform non-contextual embedders on a majority of text classification tasks. |
| Approach: | They propose to use contextual embeddings for seven languages to train new embeddables . they also show that existing embeddibles for listed languages shall be improved . |
| Outcome: | The proposed embeddings outperform non-contextual embeddables on a majority of text classification tasks. |
Revisiting Tri-training of Dependency Parsers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained word embeddings and self-training have been used in dependency parsing tasks for years. |
| Approach: | They compare tri-training and pretrained word embeddings in dependency parsing . they use language-specific FastText and ELMo embedds and multilingual BERT embedders . |
| Outcome: | The proposed methods are tri-training and pretrained word embeddings. |